Image processing apparatus, image processing system, and recording medium for programs therefor

ABSTRACT

An image processing apparatus connected to a monitoring camera and a display for processing an image input from the monitoring camera includes an image storage storing an image input from the monitoring camera, a characteristic parameter storage storing a characteristic parameter characterizing a specific image, a specific image extraction unit, wherein the specific image extraction unit cuts out images having a plurality of predetermined sizes from all parts of the input image stored in the image storage, executes character evaluation processing for checking whether or not cut out images have an identical character to the specific image for each cut out image, and extracts the cut out images which has an identical character to the specific image by the character evaluation processing, and a display image generation unit for generating a display image for displaying a whole image of the input image and the extraction specific image on the display.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the foreign priority benefit under Title 35,United States Code, §119(a)-(d) of Japanese Patent Application No.2005-352402, filed on Dec. 6, 2005, the contents of which are herebyincorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, an imageprocessing system, and a recording medium for programs therefor, whichare suitable for image processing of a monitoring image by ahigh-definition monitoring camera.

2. Description of Relevant Art

In the present day, a monitoring camera is installed at an entrance of abuilding and a parking area, at a gate of a factory site and a schoolsite, and an inside of a shop such as a large-scale store and aconvenience store, and a people flow is monitored in real time by afacility administrator or a security guard, and at the same time, themonitoring image is simultaneously stored in a storage medium such as amagnetic tape and a magnetic disk. A major purpose of the monitoring isto perceive a person displayed on the monitoring image as a mobileobject and to identify who he/she is, as well as tracking a flow line ofthe person. That is, when a suspicious person is checked using a realtime monitoring image, or when an accident occurred in the neighborhood,stored images in the past are used for finding and confirming acriminal.

Meanwhile, in recent years, with digitalization of a monitoring cameraand a progress of high-definition technology (for example, six millionpixels), a wider range of monitoring has become available, and inaddition, it has become possible to obtain personal facialcharacteristics and expressions in detail. On the other hand, since animaging capability of the high-definition monitoring camera exceeds adisplaying capability of a common display apparatus, a whole imagedmonitoring image can not be displayed on the apparatus as it is.Therefore, for displaying the whole monitoring image in one screen, theimage must be displayed by lowering the resolution. In a display of lowresolution, the personal facial characteristics and expressions can notbe displayed in detail.

Then, for example, when finding a suspicious person, or when tracking afound suspicious person, it is necessary to restore a part of the imagewhich includes, for example, a face of the suspicious person to thehigh-definition image, as needed. However, for restoring the image tothe high-definition image, a display area is required to be assigned.Since a person who should be displayed with high-definition image movesby the minute in the monitoring image, a manual operation of themonitoring camera is practically impossible.

Therefore, for eliminating the above issues, for example, an example ofa monitoring camera system, a slave camera is used for monitoring a partin detail is arranged in addition to a master camera which monitors awhole. And a specific part of the image, such as a human area and afacial area in the image is tracked by the slave camera as well asimaging the specific part of the image by zooming, is disclosed in afirst non-patent literature, Ito et al, “An cooperative IntruderSurveillance System using Master and Slave Cameras”, Symposium onrecognition and comprehension of image 2000, The Institute ofElectronics, Information and Communication Engineers, Jul. 18, 2000,p.II 229-p.II 234. In addition, an automatic detection of a human faceor the like is required for tracking a person or the like in themonitoring image. Examples of calculation methods for detecting a humanface from a monitoring image using a computer are disclosed in a secondnon-patent literature, Kazui et al, “A Study of Face Detection UsingPeripheral Increment Sign”, Workshop on practical application of avision technology, The Japan Society for Precision Engineering, Dec. 2,2004, and a third non-patent literature, Paul Viola et al, “Rapid ObjectDetection using a Boosted Cascade of Simple Features”, Proceedings ofthe 2001 IEEE Computer Society Conference on Computer Vision and PatternRecognition, IEEE Computer Society, 2001, p.I 511-p.I 518.

In the monitoring system disclosed in the first non-patent literature, acamera number corresponding to a tracking target number is required.Therefore, if the monitoring system is applied to a place where manypeoples pass through or come in and out, many slave cameras must beinstalled. In this case, a system cost becomes high, and practically,securing of an installation space for the slave cameras becomes hard.Accordingly, when a number of slave cameras as many as that of themonitoring targets can not be installed, a monitoring omission mayhappen, or a real time manual selection of the monitoring target by anobserver will be required.

In addition, when the tracking targets such as faces of many peoples areautomatically extracted from a high-definition monitoring image, since aprocessing target area is wide and the tracking targets are many, aprocessing time for the extraction becomes long. In this case, since acapability of the extraction processing is not sufficient for an inputmonitoring image which is input with a predetermined interval, a framedropping of the monitoring image may be caused. If the frame dropping iscaused, a monitoring oversight may be caused.

Meanwhile, the high-definition camera is not practically used for amonitoring camera. Therefore, there has been no proposal in the past onhow to display the high-definition monitoring image on a low capabilitydisplay apparatus and how to use the displayed image for the monitoring.

It is, therefore, an object of the present invention to provide an imageprocessing apparatus, an image processing system, and a recording mediumfor programs therefor, which can extract a monitoring target such as ahuman face from a high-definition monitoring image obtained by ahigh-definition monitoring camera within a predetermined time withoutdropping frames, and can display the extracted monitoring target as ahigh-definition image, by considering the aforementioned conventionaltechnical issues.

SUMMARY OF THE INVENTION

According to a first aspect of the present invention, there is providedan image processing apparatus connected to a monitoring camera and adisplay, which includes an image storage unit for storing an imageincluding an input image input from the monitoring camera; acharacteristic parameter storage unit for storing a characteristicparameter characterizing a specific image extracted from the inputimage; Here, a specific image is a smaller image cut out from the inputimage such as a human area and a facial area in the input image; aspecific image extraction unit, wherein the specific image extractionunit cuts out images having a plurality of predetermined sizes from allparts of the input image stored in the image storage unit, executescharacter evaluation processing for checking whether or not cut outimages have an identical character to the specific image characterizedby the characteristic parameter for each cut out image, and extracts thecut out images which are determined to have an identical character tothe specific image by the character evaluation processing as anextraction specific image; and a display image generation unit forgenerating a display image for displaying a whole image of the inputimage and the extraction specific image on the display.

In the invention, a character of the specific image to be extracted isexpressed by the characteristic parameters, and based on the parameters,an image cut out from the input monitoring image is checked whether ornot the image has a character identical to that of the specific image.Therefore, various kinds of objects, such as a male face, a female face,a child face, and a face with glasses, other than a simple face can beeasily set as a specific image to be extracted. In addition, in theimage processing apparatus according to the present invention, thedisplay image for displaying a whole image of the monitoring image andthe extraction specific image on the same display is generated.Accordingly, the observer of the display image can knowindividualistically a focused phenomenon extracted as the specific imagein detail, while obtaining the outline and key points of the phenomenonpresented from the whole monitoring image by watching the display imagewith the display.

According to a second aspect of the present invention, there is providedan image processing apparatus, wherein the characteristic parameterstorage unit stores a plurality of sets of the characteristicparameters, wherein the specific image extraction unit executes thecharacter evaluation processing for each of the plurality of sets of thecharacteristic parameters stored in the characteristic parameter storageunit, and when the cut out image is determined to have identicalcharacter to the specific image in every executed character evaluationprocessing, extracts the cut out image as the extraction specific image.

In the invention, since the image processing apparatus executes thecharacter evaluation processing between the specific images expressed bythe plurality of sets of the characteristic parameters, the specificimage can be extracted through characters obtained by multiple points ofviews.

According to a third aspect of the present invention, there is providedan image processing apparatus, wherein when a predetermined limit time,which is set in advance, elapses before completing the characterevaluation processing for all the plurality of sets of thecharacteristic parameters, the specific image extraction unit extractsthe extraction specific image based on a processing result of thecharacter evaluation processing executed before the predetermined limittime elapses.

In the invention, when the predetermined limit time, which is set inadvance, elapses by consuming the time for the character evaluationprocessing, the image processing apparatus determines whether or not thespecific image is detected based on the result of the characterevaluation processing executed before the predetermined limit timeelapses. Accordingly, the extracted specific image can be displayedwithout dropping frames even if the monitoring image is inputsequentially with a given interval.

According to a fourth aspect of the present invention, there is providedan image processing apparatus which further includes a characterevaluation processing control unit for setting a priority of theprocessing in regard to the character evaluation processing for theplurality of sets of the characteristic parameter, wherein the specificimage extraction unit executes the character evaluation processing forthe plurality of sets of the characteristic parameter based on thepriority set by the character evaluation processing control unit.

According to a fifth aspect of the present invention, there is providedan image processing apparatus which further includes a processingprogress information storage unit for storing processing progressinformation of the character evaluation processing, wherein thecharacter evaluation processing control unit stores identificationinformation of the characteristic parameter indicating that thecharacter evaluation processing is not processed in the processingprogress information storage unit when the predetermined limit timeelapses before the specific image extraction unit completes thecharacter evaluation processing for all the plurality of sets of thecharacteristic parameter and the extraction specific image is extracted,reads out the identification information of the characteristic parameterindicating that the character evaluation processing is not processedfrom the processing progress information storage unit when thepredetermined limit time remains after the specific image extractionunit completes the character evaluation processing for all the pluralityof sets of the characteristic parameter, and makes the specific imageextraction unit execute the character evaluation processing for thecharacteristic parameter identified by read out identificationinformation.

In the fourth and fifth aspects of the invention, the image processingapparatus executes the character evaluation processing according to thepriority, as well as setting the priority of the character evaluationprocessing for the plurality of sets of the characteristic parameter. Inaddition, when the processing time becomes short for the characterevaluation processing, the progress information of the processing isstored once in the processing progress information storage unit, andexecutes the remaining processing when the processing time issufficient. Accordingly, when the monitoring image is monitored in realtime, the extracted specific image is displayed on the display withoutdropping frames, and in addition, when the monitoring image is used inoff-line, the specific image which is extracted with high accuracy canbe displayed on the display.

In addition, the above-described image processing apparatus according tothe invention is characterized in that the display image generation unitgenerates a following display image. (1) The display image generationunit generates the shrunk whole image, which lowers the image quality,of the whole image of the input image in the display image. (2) Thedisplay image generation unit generates the display image where theextraction specific images are arranged at outside of the shrunk wholeimage, and further generates the lead lines connecting the part wherethe extraction specific images are extracted in the shrunk whole imageand the extraction specific images which are arranged at outside of theshrunk whole image. (3) The display image generation unit generates theshrunk whole image such that the shrunk whole image is projected on thescreen arranged to face obliquely-forward in the display image. (4) Thedisplay image generation unit arranges the extraction specific images atthe position of the part where the extraction specific images areextracted in the shrunk whole image.

In the above invention, the image processing apparatus generates thedisplay image displaying the shrunk whole image which is shrunk from thewhole image of the monitoring image by lowering the image quality andthe extracted specific image on the same display. Therefore, ahigh-definition monitoring image which is imaged by a high-definitioncamera can be displayed on a low capability display. In addition, inthis case, since a part where the specific image is extracted in theshrunk whole image and the extracted image in the part are displayed bybeing related each other with, for example, the lead line, an observerof the monitoring image can know a movement of the specific image in themonitoring image, the same way as the observer can observe the extractedspecific image in detail.

Meanwhile, the present invention includes an image processing method andprograms for operating the above-described image processing apparatuses.

Through the above, according to the present invention, it is capable ofdisplaying a monitoring image without dropping frames, displaying awhole image of the monitoring image, and displaying a high-definitionimage with respect to an important part of the image for monitoring,such as a human face.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration showing an example of a configuration of animage processing apparatus according to the first embodiment of thepresent invention;

FIG. 2 is an illustration showing an example of a display imagedisplayed on a display by the image processing apparatus according tothe first embodiment;

FIG. 3 is a flowchart showing an example of a flow of specified imageextraction processing according to the first embodiment;

FIG. 4 is an illustration showing an example of an order for moving adetection target area in the specific image extraction processingaccording to the first embodiment;

FIG. 5 is an illustration showing an example for changing an enlargementfactor of a detection target area in the specific image extractionprocessing according to the first embodiment;

FIG. 6 is a flowchart showing an example of a flow of the specific imageextraction processing when an assignment of a detection target area isimplemented with an order of “change of enlargement factor”→“movement inhorizontal axis direction”→“movement in vertical axis direction”, in thespecific image extraction processing according to the first embodiment;

FIG. 7 is a flowchart showing an example of a flow of a specific imagedetection processing according to the first embodiment;

FIG. 8 is an illustration showing an example of Haar-Wavelet-like basesused for a character extracting of an image according to the firstembodiment;

FIG. 9 is an illustration showing an example of a configuration of animage processing apparatus according to the second embodiment of thepresent invention;

FIG. 10 is an illustration showing an example of a state transitiondiagram of specific image extraction processing in the image processingapparatus according to the second embodiment;

FIG. 11 is an illustration showing an example of a configuration of abatch processing table according to the second embodiment;

FIG. 12 is a flowchart showing an example of a flow of the specificimage extraction processing according to the second embodiment;

FIG. 13 is an illustration showing an example of a flow of specificimage extraction processing according to the third embodiment of thepresent embodiment;

FIG. 14 is an illustration showing an example of a display imagedisplayed on a display by an image processing apparatus according to thefourth embodiment of the present invention;

FIG. 15 is an illustration showing an example of a modified displayimage displayed on the display by the image processing apparatusaccording to the fourth embodiment;

FIG. 16 is an illustration showing an example of a second modifieddisplay image displayed on the display by the image processing apparatusaccording to the fourth embodiment; and

FIG. 17 is an illustration showing an example of a third modifieddisplay image displayed on the display by the image processing apparatusaccording to the fourth embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT First Embodiment

Herein below, a first embodiment of the present invention will beexplained in detail by referring to FIGS. 1 to 8, as needed.

FIG. 1 is an illustration showing an example of a configuration of animage processing apparatus according to a first embodiment of thepresent invention. As shown in FIG. 1, an image processing apparatus 1is connected to a monitoring camera 2 and a display 3, and includes, forexample, an image input unit 11, an image storage unit 12, a specificimage characteristic parameter storage unit 13, a specific imageextraction unit 14, a display image generation unit 15, and an imageoutput unit 16.

The image input unit 11 receives an image signal transmitted from themonitoring camera 2, and when the image signal is an analog signal, theimage input unit 11 converts the analog signal into a digital signalusing an A/D (Analog to Digital) converter and stores the convertedimage data in the image storage unit 12. In addition, when the imagesignal is a digital signal, the image input unit 11 stores the imagedata in the image storage unit 12 after performing communicationprocessing with the monitoring camera 2 and an error correction.Meanwhile, the monitoring camera 2 is a high-definition camera with highresolution such as a high-definition spec.

The image storage unit 12 stores image data input from the monitoringcamera 2 by the image input unit 11, reads and writes the stored imagedata in response to a request of the specific image extraction unit 14and the display image generation unit 15, and changes or adds a whole ora part of the image data, as needed. Meanwhile, in the specification,“image data” may be simply written as “image” in some case if there isno possibility to be mistaken.

The specific image characteristic parameter storage unit 13 stores acharacteristic parameter which is necessary for detecting a specificimage, that is, a monitoring target, for example, a human face. Thespecific image characteristic parameter is calculated in advance from apredetermined teacher data and a learning program before operating theimage processing apparatus 1, and stored in the specific imagecharacteristic parameter storage unit 13.

The specific image extraction unit 14 evaluates whether or not an imagehas an identical character to that of a specific image by using thespecific image characteristic parameter stored in the specific imagecharacteristic parameter storage unit 13, for all arbitrary area ofinput image data input from the monitoring camera 2 and stored in theimage storage unit 12. If the image has the identical character, thespecific image extraction unit 14 extracts the image data, and storesthe data in the image storage unit 12. Meanwhile, processing of thespecific image extraction unit 14 in detail will be described later.

The display image generation unit 15 generates a shrunk image which islowered a resolution so that a whole image of the image input from themonitoring camera 2 can be displayed on a display screen of the display3. Then, the display image generation unit 15 generates a display imagefor displaying both the shrunk image of the whole image and the imagehaving the identical character to that of the specific image which isextracted by the specific image extraction unit 14 on the display screenof the display 3. Meanwhile, an example of the display image will bedescribed later in detail by referring to FIG. 2.

The image output unit 16 outputs display image data generated by thedisplay image generation unit 15 to the display 3 based on apredetermined interface spec. Here, the display 3 is, for example, a LCD(Liquid Crystal Display) display and a CRT (Cathode Ray Tube) display,and a higher display capability (for example, a pixel number) ispreferable, but it does not matter if not higher.

The above-described image processing apparatus 1 is configured with acomputer which includes a CPU (Central Processing Unit) composed of, forexample, an arithmetic device (not shown) and a storage unit, such as asemiconductor memory and a hard disk storage unit. In this case, anentity of the image storage unit 12 and specific image characteristicparameter storage unit 13 is a storage area assigned to a part of thestorage unit. In addition, the specific image extraction unit 14 and thedisplay image generation unit 15 are actualized by the CPU executingpredetermined programs stored in the storage unit. Also, the image inputunit 11 and the image output unit 16 are actualized by using aninterface circuit for the input or the output and by executing programsfor driving and controlling the interface circuit by the CPU.

FIG. 2 is an illustration showing an example of a display imagedisplayed on a display by an image processing apparatus according to thefirst embodiment of the present invention. As shown in FIG. 2, on adisplay image 31 displayed on a screen of the display 3, a shrunk wholeimage 32 which is shrunk from a whole high-definition monitoring imageinput from the monitoring camera 2 by lowering the resolution and aspecific image 33 extracted by the specific image extraction unit 14 aredisplayed. In this case, the shrunk whole image 32 is arranged atapproximately a center of the display image 31, and the extractedspecific image 33 is arranged around the shrunk whole image 32. Inaddition, the extracted specific image 33 is displayed without loweringthe resolution. Meanwhile, “the extracted specific image 33 is displayedwithout lowering the resolution” means that “the extracted specificimage 33 is displayed without lowering the resolution as much as theshrunk whole image 32”. A resolution of the specific image 33 may belowered as needed within a practical range according to a monitoringpurpose.

In FIG. 2, a specific image of an extraction target is a human face.Therefore, a human face is extracted from a whole image of a monitoringimage, and the extracted human face, that is, the extracted specificimage 33 is displayed around a shrunk whole image 32 without loweringthe resolution. Accordingly, an observer can see facial characteristicsand expressions in detail as fine as can be used for a monitoringpurpose from the human face displayed without lowering the resolution.

In addition, in FIG. 2, a lead line 34 which connects the specific image33 (a human face) and a face portion of a person in the shrunk wholeimage 32 is displayed so that it can be easily seen which person in theshrunk whole image 32 corresponds to the extracted human face. Further,since the shrunk whole image 32 of the monitoring image is displayed inthe display image 31 displayed on the display 3 in FIG. 2, a flow lineof each person can be monitored, looking at the display image 31 at agiven interval.

Meanwhile, in FIG. 2, a space under eaves of a house and a human footare shown as extracted noise examples by an erroneous detection of thespecific image in the specific image extraction unit 14.

Next, specific image extraction processing to be executed by thespecific image extraction unit 14 will be explained in detail byreferring to FIGS. 3 to 8. Here, FIG. 3 is a flowchart showing anexample of a flow of the specific image extraction processing accordingto the first embodiment of the present invention. FIG. 4 is anillustration showing an example of an order for moving a detectiontarget area in the specific image extraction processing. FIG. 5 is anillustration showing an example for changing an enlargement factor inthe specific image extraction processing.

In the specific image extraction processing, the specific imageextraction unit 14 cuts out an area from an input image, which is inputfrom the monitoring camera 2 and stored in the image storage unit 12, asa specific image detection target area, and checks whether or not acharacter of the cut out image is identical to that of a specific image(for example, a human face) which is set in advance. If determined to beidentical, the cut out image is stored in the image storage unit 12 asan extraction specific image.

As shown in FIG. 3, the specific image extraction processing of thespecific image extraction unit 14 starts when the image input unit 11inputs image data from the monitoring camera 12 and stores the imagedata in the image storage unit 12. Then, when the specific imageextraction processing starts, the specific image extraction unit 14(that is, CPU (not shown) in image processing apparatus 1) first sets adetection target area 42 (refer to FIG. 4) for detecting a specificimage in the input image (step S11). The detection target area 42 has,as shown in FIG. 4, a predetermined size and set at, for example, upperleft of an input image 41.

When the detection target area 42 is set, by skipping steps S12 to S14at only first time, an image of the detection target area 42 is cut out,and the specific image detection processing is executed (step S15).Meanwhile, the specific image detection processing will be explainedlater in detail by referring to FIGS. 7 and 8.

When the specific image detection processing for the set detectiontarget area 42 is completed, next, a detection target area 42 is resetin another area of the input image. Here, resetting of the detectiontarget area 42 is, as shown in FIG. 4, repeated until the detectiontarget area 42 reaches a right end by moving the detection target area42 in a horizontal axis direction bit by bit (horizontal axis directionscan). In addition, when one of the horizontal axis direction scans iscompleted, another horizontal axis direction scan is repeated again bylowering a vertical axis position of the detection target area 42 a bit,and repeats the scan until a vertical axis position of the detectiontarget area 42 reaches the bottom position of the input image 41(vertical axis direction scan).

That is, in FIG. 3, the specific image extraction unit 14 checks whetheror not the specific image detection processing is completed for allhorizontal axis positions after completing the specific image detectionprocessing (step S16), and if the processing is not completed for theall horizontal axis positions (step 16: No), the processing returns tostep S14 and executes the specific image detection processing again(step S15) by moving the detection target area 42 to the horizontal axisdirection (step S14). In addition, when the specific image detectionprocessing is completed for all horizontal positions (step 16: Yes), thespecific image extraction unit 14 checks whether or not the specificimage detection processing is completed for all vertical axis positions(step S17). If the specific image detection processing is not completedfor the all vertical axis positions (step 17: No), the processingreturns to step S13, and executes the steps S14 to S16 again by movingthe detection target area 42 to the vertical axis direction (step S13).In addition, from the check at the step S17, if it is found that thesteps S14 to S17 are completed for the all vertical axis positions (stepS17: Yes), the horizontal and vertical axis direction scans of thedetection target area 42 with a given size and the specific imagedetection processing at each position are completed.

Then, as shown in FIG. 5, by enlarging or shrinking the detection targetarea 42 bit by bit and by repeating the horizontal axis direction scanand the vertical axis direction scan of the detection target area 42with each enlargement factor, the specific image detection processing isexecuted.

That is, when the specific image extraction unit 14 completes thehorizontal axis direction scan, vertical axis direction scan, andspecific image detection processing at each position of the detectiontarget area 42 with a given size (step S17: Yes), subsequently, thespecific image extraction unit 14 checks whether or not the specificimage detection processing is completed for all enlargement factors(step S18). If the specific image detection processing is not completedfor the all enlargement factors (step S18: No), the processing returnsto the step S12, and executes the steps S12 to S17 again by changing theenlargement factor of the detection target area 42 (step S12). If thespecific image detection processing is completed for all the enlargementfactors (step S18: Yes), the specific image detection processing ends.

Here, regarding the enlargement factor of the detection target area 42,it is not necessary to set every enlargement factors acceptable in theinput image 41. The enlargement factor may be sufficient if it is in apractical range by considering a size of the specific image which isimaged in the input image 41. In addition, here, the setting of thedetection target area 42 is executed in the following order. “Move asearch window in horizontal axis direction”→“Move a search window invertical axis direction”→“change enlargement factor”. However, the orderis not limited thereto. FIG. 6 is a flowchart showing an example of aflow of the specific image extraction processing when the setting ofdetection target area is executed in the following order. “changeenlargement factor”→“Move a search window in horizontal axisdirection”→“Move a search window in vertical axis direction”.

A difference between the processing flow shown in FIG. 6 and that ofshown in FIG. 3 is that the processing for changing the enlargementfactor (step S12) and the processing for checking whether or not thespecific image detection processing is completed for all enlargementfactors (step S18) are set inside the repeating processing of the stepsS14 to S16. Because of the above, the setting of the detection targetarea 42 is executed by the following order. “change enlargementfactor”→“Move a search window in horizontal axis direction”→“Move asearch window in vertical axis direction”.

Meanwhile, in principle, an extraction result of a specific image is notchanged by changing an order of the setting of the detection target area42. However, practically, the extraction result may be changed a bit,for example, by a moving step value and a step value of the enlargementfactor, but not much.

Next, the specific image detection processing shown in FIG. 3 will beexplained in detail by referring to FIGS. 7 and 8. Here, FIG. 7 is aflowchart showing an example of a flow of the specific image detectionprocessing according to the first embodiment of the present invention.FIG. 8 is an illustration showing an example of Haar-Wavelet-Like basesto be used for extracting an image.

As shown in FIG. 7, the specific image detection processing according tothe embodiment is configured to include a specific image evaluationprocessing unit 141 which is formed by multiple stacking of unitprocessing composed of a characteristic quantity calculation unit 142and a character identification unit 143. Hereinafter, the unitprocessing composed of the characteristic quantity calculation unit 142and the character identification unit 143 is called as a characterevaluation processing unit 144. Therefore, in the case of FIG. 7, thecharacter evaluation processing unit 144 at a first step is composed ofF101 and C101, that of a second step is composed of F102 and C102, andthat of a third step is composed of F103 and C103. That is, in the caseof FIG. 7, a specific image is detected by executing the characterevaluation processing three times. Meanwhile, in FIG. 7, a stackingnumber of the unit processing in the character evaluation processingunit 144 is three steps, however, any number of the step no less thanone is available.

The characteristic quantity calculation unit 142 calculates acharacteristic quantity Fj which is formulated by formula 1.

$\begin{matrix}{F_{j} = {\sum\limits_{i}( {{\alpha_{j,i}{\sum\limits_{{({x,y})} \Subset S_{i,{white}}}I_{x,y}}} - {\sum\limits_{{({x,y})} \Subset S_{i,{black}}}I_{x,y}}} )}} & ( {{formula}\mspace{14mu} 1} )\end{matrix}$

In the formula 1, j is an identification number of the characteristicquantity calculation unit 142 (that is, the character evaluationprocessing unit 144). When the character evaluation processing unit 144is stacked as shown in FIG. 7, an order of the character evaluationprocessing unit 144 from the top thereof may be assigned to theidentification number. In addition, “Si,white” and “Si,black” indicate awhite area and a black area respectively in the Haar-Wavelet-Like bases(refer to FIG. 8) with an identification number i. α_(j,i) is aparameter indicating a character of a specific image for theHaar-Wavelet-Like bases with the identification number i in thecharacteristic quantity calculation unit 142 with the identificationnumber j. Also, I_(x,y) indicates a luminance value at coordinates (x,y) in a detection target area.

Meanwhile, the Haar-Wavelet-Like bases are, as shown in FIG. 8, forexample, fourteen graphical primitive images composed of a white areaand a black area, and used for expressing a character of a specificimage of an extraction target such as a human face. Here, theidentification number i (i=1, 2, . . . , 14) is labeled to eachHaar-Wavelet-Like bases in FIG. 8. Meanwhile, a detailed explanation onthe Haar-Wavelet-Like bases is described in the third non-patentliterature.

In the embodiment, it is assumed that a specific image to be extractedis characterized by a set of parameters (αj,1, αj,2, . . . , αj,14)based on the Haar-Wavelet-Like bases. In this case, for expressing acharacter of a specific image having vagueness such as a human face, aplurality of sets of parameters (αj,1, αj,2, . . . , αj,14) (j=1, 2, . .. , N) are required in general, and each set of parameters correspondsto each characteristic quantity calculation units 142 in FIG. 7. Here, Nis a set number of parameters, or a characteristic quantity calculationunit 142 number included in the specific image evaluation processingunit 141.

Next, the character identification unit 143 calculates formula 2, andbased on the result, checks whether or not an image in a detectiontarget area has a character identical to that of a specific image.

$\begin{matrix}{{Object} = \{ \begin{matrix}{1\text{:}\mspace{14mu}( {F_{j} \geq \theta_{j}} )} \\{0\text{:}\mspace{14mu}( {F_{j} < \theta_{j}} )}\end{matrix} } & ( {{formula}\mspace{14mu} 2} )\end{matrix}$

Here, θj is a threshold value set in advance in the characteridentification unit 143 which forms a pair together with thecharacteristic quantity calculation unit 142 with identification numberj. According to the formula 2, a value of the “Object” becomes 1 whenthe characteristic quantity Fj, which is calculated by thecharacteristic quantity calculation unit 142, is no less than thethreshold value θj, and when less than the θj, the “Object” becomes 0(zero). When the “Object” is 1, the character identification unit 143determines that the image in the detection target area has the identicalcharacter. On the other hand, when the “Object” becomes 0 (zero), thecharacter identification unit 143 determines that the image in thedetection target area does not have the identical character, that is,the character identification unit 143 determines that the image is notthe specific image to be detected.

As described above, the character evaluation processing unit 144(characteristic quantity calculation unit 142 and characteridentification unit 143) checks whether or not an image in a detectiontarget area has a character identical to that of a specific image, byusing the set of parameters (αj,1, αj,2, . . . , αj,14) which expressesa character of a specific image corresponding to the characterevaluation processing unit 144, and the threshold value θj (j=1, 2, . .. ). In FIG. 7, arrows leaded out to right sides from rhombic blocks(C101, C102, C103) of the character identification unit 143 indicatethat an image in the detection target area is determined to have acharacter identical to that of a specific image, and arrows lead outdownward indicate that the image in the detection target area does nothave a character identical to that of the specific image.

That is, when all character evaluation processing units 144 included inthe specific image evaluation processing unit 141 determine that,namely, when all character identification units 143 (C101, C102, C103)determine that “the image in the detection target area has a characteridentical to that of the specific image”, the specific image evaluationprocessing unit 141 determines that “the specific image is detected”. Inaddition, when any one of the character identification units 143 (C101,C102, C103) determine that “the image in the detection target area doesnot have a character identical to that of the specific image”, thespecific image evaluation processing unit 141 determines that “thespecific image is not detected”.

From the above, when the specific image evaluation processing unit 141determines that “the specific image is detected”, the specific imageextraction unit 14 extracts the image in the detection target area as anextraction specific image, stores the extracted extraction specificimage in the image storage unit 12, and ends processing of the specificimage evaluation processing unit 141. On the other hand, when thespecific image evaluation processing unit 141 determines that “thespecific image is not detected”, the specific image extraction unit 14directly ends the processing of the specific image evaluation processingunit 141.

Meanwhile, the set of parameters (αj,1, αj,2, . . . , αj,14) and thethreshold value θj (j=1, 2, . . . , N) which are used in the aboveprocessing are calculated in advance before executing processing of thespecific image evaluation processing unit 141, by using, for example, apredetermined learning program and teacher data, and the calculatedvalues are stored in the specific image characteristic parameter storageunit 13.

Subsequently, the display image generation unit 15 (refer FIG. 1)displays the specific image 33, which is extracted through theabove-described manner and stored in the image storage unit 12, on thedisplay 3 as a display image as shown in FIG. 2, together with theshrunk whole image 32.

As described above, according to the first embodiment of the presentinvention, for example, by automatically extracting a specific imagesuch as a human face by using a computer from an input image inputthrough the monitoring camera 2 for, for example, monitoring or thelike, the extracted extraction specific image such as a human face isdisplayed around a shrunk whole image of the input image withoutlowering the resolution. Therefore, from the displayed specific imagesuch as a human face, for example, the characteristics and expressionsof the specific image can be obtained. In addition, a movement of thespecific image such as a human face which has a certain character, thatis, a flow line of the specific image can be obtained within a wholemonitoring image.

Second Embodiment

Hereinafter, a second embodiment of the present invention will beexplained by referring to FIGS. 9 to 12.

FIG. 9 is an illustration showing an example of a configuration of animage processing apparatus according to a second embodiment of thepresent invention. As shown in FIG. 9, an image processing apparatus 1 aaccording to the second embodiment has a configuration where a characterevaluation processing control unit 17 and a processing progressinformation storage unit 18 are added to the configuration of the imageprocessing apparatus 1 (refer to FIG. 1) according to the firstembodiment. That is, the second embodiment has a configuration identicalto that of the first embodiment except the character evaluationprocessing control unit 17 and the processing progress informationstorage unit 18. Then, in the second embodiment, a same symbol islabeled to a component identical to that of the first embodiment, and anexplanation thereof will be omitted.

The character evaluation processing control unit 17 controls aprocessing quantity of the specific image evaluation processing unit 141(refer to FIG. 7) in the specific image extraction unit 14. In addition,the processing progress information storage unit 18 stores progressinformation of processing when the processing is interrupted by thecharacter evaluation processing control unit 17 for controlling theprocessing quantity of the specific image evaluation processing unit141.

In the embodiment, a monitoring image is input from the monitoringcamera 2 in real time with a predetermined interval, and a predeterminedspecific image is extracted from the input image. Then, extractionprocessing of the specific image is required to be completed within apredetermined time. However, when the character evaluation processingunit 144, which is composed of the character calculation unit 142 andthe character identification unit 143, is multiply stacked in thespecific image evaluation processing unit 141, in addition, when manyspecific images of extraction targets exist in the input image, a longtime is required for extracting the specific images. As a result,extractions of the specific images for all areas of the input image cannot be completed within the predetermined time in some case. Then, byreducing the processing quantity of the specific image evaluationprocessing unit 141 in the specific image extraction unit 14, thecharacter evaluation processing control unit 17 eliminates the aboveissue that the extractions of the specific images for all areas of theinput image can not be completed. Herein below, the details will beexplained.

As explained in the first embodiment, the specific image evaluationprocessing unit 141 reads out a set of parameters (αj,1, αj,2, . . . ,αj,14) characterizing a specific image to be extracted and its thresholdvalue θj from the specific image characteristic parameter storage unit13, and determines whether the specific image is detected (Detection) ornot detected (Non-detection), by operating each character evaluationprocessing unit 144 corresponding to each parameter by each set ofparameters.

Here, a point to notice is that each character evaluation processingunit 144 corresponding to each set of parameters has a differentdetection capability and different processing time of the specific imageto each other. In addition, in the first embodiment, whether thespecific image is detected (Detection) or not detected (Non-detection)is determined after operating all character evaluation processing units144 stacked in the specific image evaluation processing unit 141.However, in the second embodiment, whether the specific image isdetected or not detected is determined based on a determination resultof an executed character evaluation processing unit 144 withoutoperating all the stacked character evaluation processing units 144. Inthis case, the issue is not to become unable to detect a specific image,but lowering of a detection capability of the specific image.

Therefore, the character evaluation processing control unit 17 evaluatesa processing time and detection capability of the character evaluationprocessing units 144 in the specific image evaluation processing unit141, and based on the evaluation, determines a priority and processingnumber of the processing so that a specific image can be detectedefficiently within a predetermined limit time.

In addition, the character evaluation processing control unit 17implements a progress management of the character evaluation processingunits 144 in the specific image extraction processing. In the progressmanagement, when the predetermined limit time has elapsed before everyprocessing of the character evaluation processing units 144 in thespecific image evaluation processing unit 141 is completed, whether aspecific image is detected or not detected is determined based onevaluation results implemented by the character evaluation processingunits 144 before the predetermined limit time has elapsed. In addition,further, the progress information (information of, for example,processing completed/unprocessed) of the character evaluation processingunits 144 in the specific image evaluation processing unit 141 is storedin the processing progress information storage unit 18. On the otherhand, when the processing of the character evaluation processing units144 in the specific image evaluation processing unit 141 is completedand if a time remains until the predetermined limit time elapses,unprocessed processing of the character evaluation processing units 144stored in the processing progress information storage unit 18 isexecuted.

FIG. 10 is an illustration showing an example of a state transition ofspecific image extraction processing in an image processing apparatusaccording to the second embodiment. In FIG. 10, a black circle indicatesa starting state, and a double black circle indicates an ending state.First, the image processing apparatus 1 a transits to an image inputstate (S21) triggered by an input signal from the monitoring camera 2,and the image input unit 11 inputs image data input from the monitoringcamera 2 and stores the input image data in the image storage unit 12.

If storage of the input image data in the image storage unit 12 iscompleted, the image processing apparatus 1 a transits to an imageprocessing state (S22). In the image processing state (S22), the imageprocessing apparatus 1 a alternately transits between a processing planstate (S23) and a processing execution state (S24) after everyprocessing completion of the each state until predetermined specificimage extraction processing for the input image is completed, or thepredetermined limit time elapses.

The character evaluation processing control unit 17 of the imageprocessing apparatus 1 a evaluates a processing time and a detectioncapability of a specific image with respect to each character evaluationprocessing units 144 in the specific image evaluation processing unit141, as the processing of the processing plan state (S23), and plans,for example, which processing of the character evaluation processingunits 144 should be preferentially executed. In addition, the specificimage extraction unit 14 executes predetermined processing of thespecific image evaluation processing unit 141, based on an executionplan of processing planned at the processing plan state (S23), as theprocessing of the processing execution state (S23). Meanwhile, aprocessing content of the processing plan state (S23) will be describedlater.

When the predetermined limit time has elapsed, the specific imageextraction unit 14 ends processing of the specific image evaluationprocessing unit 141 even if it is before completion of processing of theprocessing execution state (S24), and transits to a batch registrationstate (S25) Then, in the batch registration state (S25), progressinformation of the processing is registered in a batch processing table(refer to FIG. 11) so that unprocessed processing of the specific imageevaluation processing unit 141 can be restarted later, and thepredetermined processing of the specific image evaluation processingunit 141 for an image of the detection target area is ended.

FIG. 11 is an illustration showing an example of a configuration of abatch processing table according to the second embodiment. In FIG. 11,an image data number is a number for identifying image data of adetection target area. In addition, in the batch processing table,information such as a completed processing content, a remainingprocessing content, and an estimated time for completing processing, ofthe specific image evaluation processing unit 141 for the image data isstored so as to correspond to the image data number.

Here, the explanation returns to FIG. 10. When the predeterminedprocessing is completed at the processing execution state (S24), thatis, when every processing of the specific image evaluation processingunit 141 planned at the processing plan state (S23) is completed, iftime remains before the predetermined limit time elapses, the imageprocessing apparatus 1 a transits to a registered task fetch state(S26). In the registered task fetch state (S26), the image processingapparatus 1 a, that is, the specific image extraction unit 14 fetchesone of registered batch processing tasks from the batch processing tableof the processing progress information storage unit 18, and transits tothe processing execution state (S22). Then, in the processing executionstate (S22), the fetched registered batch task is executed as with theprocessing for the input image.

FIG. 12 is a flowchart showing an example of a flow of specific imageextraction processing according to the second embodiment. As shown inFIG. 12, when an image is input from the monitoring camera 2, the imageprocessing apparatus 1 a plans a procedure of specific image extractionprocessing (step S31), for example, for determining a priority (order ofprocessing) and processing number of processing of the characterevaluation processing units 144, based on, for example, a processingtime, a specific image detection capability, and time allowable forprocessing each character evaluation processing units 144 in thespecific image evaluation processing unit 141 (refer to FIG. 7).

Hereinafter, since processing of steps S13 to S18 is the same with thatof shown in FIG. 3, the explanation will be omitted. However, in theembodiment, all character evaluation processing units 144 included inthe specific image evaluation processing unit 141 are not executed inthe specific image detection processing at the step S15, but executedaccording to a processing procedure planned at the step S31.

If the predetermined detection processing planned at the step S31 iscompleted for every areas of an input image by the step S18, thecharacter evaluation processing control unit 17 checks whether or notthere remains a processing time with respect to the predetermined limittime (step S32). As a result of the check, if there remains theprocessing time (step S32: Yes), whether or not every processing of thecharacter evaluation processing units 144 included in the specific imageevaluation processing unit 141 is completed, that is, whether or notthere remains detection processing, is checked (step S35). As a resultof the above check, if there remains the detection processing (characterevaluation processing units 144, same in below) to be executed in thespecific image evaluation processing unit 141 (step S35: Yes), the stepreturns to the step S31, and the steps after the step S12 are executedby planning the detection procedure again for the remaining detectionprocessing.

In addition, in the check at the step S35, if there remains no detectionprocessing to be executed in the specific image evaluation processingunit 141 (step S35: No), processing which is determined as remainingdetection processing (unprocessed) of the character evaluationprocessing units 144, that is, detection processing registered as abatch processing task in the batch processing table (refer to FIG. 11),is taken out from the batch processing table, and executed (step S36).

In addition, in the check at the step S32, when there remains noprocessing time (step S32: No), whether or not every detectionprocessing of the specific image evaluation processing unit 141 isexecuted, that is, whether or not there remains detection processing(step S33) is checked. As a result of the check, if there remainsdetection processing to be executed in the specific image evaluationprocessing unit 141 (step S33: Yes), the remaining detection processingis registered in the batch processing table as a batch processing task(step S34), and extraction processing of a specific image is ended. Inaddition, if there remains no detection processing to be executed in thespecific image evaluation processing unit 141 (step S33: No), theextraction processing of the specific image is directly ended.

At the step S15 explained in the above, if an image in the detectiontarget area is determined to have identical characters with that of aspecific image in all checking results (character identification unit143) of the detection processing planed at the step S31, it isdetermined that the specific image is detected, and the image in thedetection target area is stored in the image storage unit 12 as anextraction specific image. However, when the image in the detectiontarget area is determined not to be the specific image in the subsequentprocessing, the image in the detection target area once stored in theimage storage unit 12 is deleted.

Next, processing (step S31 in FIG. 12) in the processing plan state (S23in FIG. 10) will be explained. In processing of the processing plan, aprocessing time and detection capability of the character evaluationprocessing unit 144 are evaluated in advance.

When the Haar-Wavelet-Like bases (refer to FIG. 8) are applied to acharacteristic quantity calculation, a processing time of the charactercalculation unit 142 becomes long in proportion to an image size of thebases, that is, a memory access time. Also, if the base numberincreases, the processing time becomes long. Therefore, the processingtime can be estimated from the size and a base number. Further, sinceonly an area where a specific image is not detected in an input imagebecomes a target area for the specific image detection processing, theprocessing time also becomes long in proportional to a size of thisarea.

Accordingly, an estimated processing time Tj of the character evaluationprocessing unit 144 with an identification number j is expressed informula 3.

$\begin{matrix}{T_{j} = {R_{j}( {{w_{j,1} \cdot {\sum\limits_{i = 1}^{b_{j}}s_{j,i}}} + {w_{j,2} \cdot b_{j}}} )}} & ( {{formula}\mspace{14mu} 3} )\end{matrix}$

In the formula 3, s_(j,i) is a pixel number relating to a base, which isincluded in the characteristic quantity calculation unit 142, with anidentification number i, b_(j) is a base number which is included in thecharacteristic quantity calculation unit 142 with an identificationnumber j, Rj is an area of an image of a detection target, and wj,1,wj,2 are proportionality coefficients.

In addition, a detection capability Aj of a specific image of the unitprocessing composed of the j-th characteristic quantity calculation unit142 and character identification unit 143 is, for example, defined byformula 4.

$\begin{matrix}{A_{j} = \frac{( {{detected\_ number}{\_ by}{\_ character}{\_ identification}{{\_ unit}/{correct\_ non}}\text{-}{detected\_ number}} )}{( {{teacher\_ image}{\_ data}{\_ number}{\_ input}{\_ in}{\_ characteristic}{\_ quantity}{\_ calculation}{\_ unit}} )}} & ( {{formula}\mspace{14mu} 4} )\end{matrix}$

Meanwhile, the detection capability Aj can be learnt together whenparameters (αj1, αj,2, . . . , αj,14) and a threshold value θj (j=1, 2,. . . ) corresponding to the parameters are learnt from a predeterminedteacher data.

Further, in processing for planning the detection processing (step S31in FIG. 12), unit processing of the detection processing which has asuperior detection efficiency of a specific image is preferentiallyselected and executed by trading off the estimated processing time Tjand detection capability Aj. The unit processing of detection processingfor preferential processing is selected based on formula 5.arg min_(j)(T_(j)+α/A_(j))  (formula 5)

In formula 5, j is an identification number for identifying unitprocessing composed of the characteristic quantity calculation unit 142and character identification unit 143 (j=1, 2, . . . , N: N is a unitprocessing number included in the specific image evaluation processing141). Also, α is a weighting coefficient for determining tradeoffbetween a processing time and a detection capability, and a function“arg min” is a function for calculating j with which a value of formulawithin the case arc becomes minimum.

Meanwhile, processing for planning detection processing is not limitedto use the formula 5, but various kinds of modifications of the formula5 are possible. For example, in consideration of a predetermined limittime in advance, a plan for increasing a response may be available byselecting unit processing having a superior detection capability in thefirst half of the limit time even if a long time is required for theprocessing, and selecting unit processing having a short processing timein the second half of the limit time. The above can be achieved, asshown in formula 7, by defining that the weighting coefficient α in theformula 5 varies depending on an elapsed time t within the limit time T.In this case, the formula 5 is expressed with formula 6.

$\begin{matrix}{{\arg\min}_{j}( {T_{j} + {{\alpha(t)}/A_{j}}} )} & ( {{formula}\mspace{14mu} 6} ) \\{{\alpha(t)} = \{ \begin{matrix}{1\text{:}\mspace{14mu}( {{{if}\text{:}\mspace{14mu} t} \leq {\frac{3}{4}T}} )} \\{10\text{:}\mspace{14mu}( {{{if}\text{:}\mspace{14mu} t} > {\frac{3}{4}T}} )}\end{matrix} } & ( {{formula}\mspace{14mu} 7} )\end{matrix}$

As described above, according to the second embodiment, the imageprocessing apparatus 1 a plans a processing procedure for specific imageextraction processing in consideration of a predetermined limit time inadvance, and based on the processing procedure, further executesextraction processing of a specific image, while implementing a progressmanagement of the detection processing. Therefore, for example, afollowing disadvantage never happens that an extraction image of aspecific image can not be obtained due to shortage of time. Accordingly,for example, when an image from the monitoring camera 2 is monitored inreal time, frame dropping of an extraction image of a specific imagenever happens even if the limit time of the specific image extractionprocessing is short.

In addition, extraction processing of a specific image which has notbeen executed due to shortage of time is executed when there is aremaining time squeezed out through processing of other input image.That is, when a monitoring image is used in off-line, for example, forverifying a crime, extraction of the specific image can be executed forall detection conditions (set of parameters characterizing a specificimage) set at the beginning. Therefore, in this case, a monitoring imagedisplaying a specific image extracted under the best detection conditioncan be provided.

Third Embodiment

Hereinafter, a third embodiment of the present invention will beexplained by referring to FIG. 13.

FIG. 13 is an illustration showing an example of a flow of specificimage detection processing according to a third embodiment of thepresent invention. In the embodiment, a plurality (in the example,three) of specific image evaluation processing units which are identicalto the specific image evaluation processing unit 141 shown in FIG. 7 aredisposed. In each of the specific image evaluation processing units 141a, 141 b, and 141 c, a different specific image is detectedrespectively. For example, when a human face is detected, each of thespecific image evaluation processing units 141 a, 141 b, and 141 cdetects independently a full-faced human face, a diagonallyforward-faced human face, and a side-faced human face, respectively.

In FIG. 13, processing within the each specific image evaluationprocessing units 141 a, 141 b, and 141 c is executed with a similarmanner to that of the specific image evaluation processing unit 141shown in FIG. 7. In addition, in FIG. 13, when a first specific image(for example, a full-faced human face) is detected by executing thespecific image evaluation processing units 141 a, an image of adetection target area at the time is stored in the image storage unit12, and when the first specific image is not detected, the specificimage evaluation processing units 141 b detects a second specific image(for example, a diagonally forward-faced human face). If the secondspecific image is detected, an image of a detection target area at thetime is stored in the image storage unit 12. As with a manner describedabove, a third specific image (for example, a side-faced human face) isdetected.

Meanwhile, in each character evaluation processing unit 144 of thespecific image evaluation processing units 141 a, 141 b, and 141 c, aset of parameters (αj,1, αj,2, . . . , αj,14) indicating a character ofa specific image and a threshold value θj (j=1, 2, . . . , N)corresponding to the set of parameters are set. Values of theseparameters are calculated in advance by, for example, a predeterminedprogram and teacher data, and the calculated values are stored in thespecific image characteristic parameter storage unit 13.

In addition, in FIG. 13, by further adding a character evaluationprocessing unit similar to the specific image evaluation processingunits 141 a, 141 b, and 141 c, a male face, a female face, a child face,and the like may be detected by each added character evaluationprocessing unit. Further, for example, an age level, a hair style, askin color, a hair color, a head shape, with or without glasses, with orwithout mask, with or without accessories, with or without lentigo, awhisker shape, and a whisker color may be detected by configuring aswith the above.

Meanwhile, when a human face is detected by differentiating an agelevel, a hair style, a skin color, a hair color, a head shape, with orwithout glass, with or without mask, with or without accessories, withor without lentigo, a whisker shape, and a whisker color, all specificimage evaluation processing units are not executed in parallel, butprocessing for detecting a human face is executed once, and only when ahuman face is detected, an age level, a hair style, a skin color, a haircolor, a head shape, with or without glass, with or without mask, withor without accessories, with or without lentigo, a whisker shape, and awhisker color may be detected. In this case, a configuration of a flowof specific image detection processing includes a configuration stackedin series, as well as parallel configuration of the specific imageevaluation processing units (141 a, 141 b, 141 c) as shown in FIG. 13.

In addition, as a modified configuration of the third embodiment, aconfiguration where each specific image evaluation processing units 141a, 141 b, and 141 c is executed by each independent computer may beavailable. In this case, in the image processing apparatus 1 in FIG. 1,a processing distribution control unit for distributing specific imageextraction processing is arranged at a position of the specific imageextraction unit 14, and a plurality of computers are arranged under theprocessing distribution control unit and each specific image evaluationprocessing units 141 a, 141 b, and 141 c of the specific imageextraction unit 14 is arranged to each computer. In this case, a body ofthe image processing apparatus 1 including the processing distributioncontrol unit and the plurality of computers under the processingdistribution control unit are connected with a network, for example, aprivate line or LAN (Local Area Network). In this configuration, sincethe extraction processing of the specific image is executedsimultaneously by the plurality of computers, a processing time for theextraction processing can be substantially shortened.

Fourth Embodiment

Subsequently, as a fourth embodiment of the present invention, anexample of a display image displayed on the display 3 by the imageprocessing apparatuses 1, 1 a described in the first to thirdembodiments will be shown by referring FIGS. 14 to 17. Here, FIG. 14 isan illustration showing an example of a display image displayed on adisplay by an image processing apparatus according to a fourthembodiment of the present invention. Also, FIG. 15 is a modified exampleof the display image example in FIG. 14, FIG. 16 is a second modifiedexample of the display image example in FIG. 14, and FIG. 17 is a thirdmodified example of the display image example in FIG. 14, respectively.

Meanwhile, a configuration of the present embodiment is identical tothat of, for example, the first embodiment except the display imagegeneration unit 15 (refer to FIGS. 1, 9). In addition, a display imagegeneration function for displaying the shrunk whole image 32 as a wholeimage by lowering a resolution of a monitoring image by the displayimage generation unit 15 and for displaying the specific image 33extracted by the specific image extraction unit 14 without lowering theresolution is identical to that of the first embodiment.

In the example of the display image 31 shown in FIG. 2, the specificimage 33 extracted by the specific image extraction unit 14 is arrangedat outside of four sides of the shrunk whole image 32. In that case,there is a possibility that the lead lines 34 may cross to each otherand may pass a specific image detection part (a human face part) of theshrunk whole image 32. If the lead lines 34 cross to each other, or passanother specific image detection part, the display image 31 becomesdifficult to see clearly.

Therefore, in this embodiment (a part except the display imagegeneration unit 15 is the same with that of any one of embodimentsaccording to the embodiments 1 to 3), the display image generation unit15 generates, as shown in FIG. 14, the display image 31 a so that aplurality of rows of extracted specific images 33 a can be arrangedaround a periphery of each four sides of the shrunk whole image 32 a. Inthis case, since a freedom of arrangement of the extracted specificimages 33 a is increased, the possibility that the lead lines 34 crossto each other, or pass another specific image detection part (a humanface) can be reduced.

In addition, as shown in FIG. 15, the display image generation unit 15may arrange the extracted specific images 33 a, for example, around aperiphery of three sides of the shrunk whole image 32 b except thebottom side, instead of arranging around a periphery of all four sides.

In addition, as shown in FIG. 16, the display image generation unit 15displays a shrunk whole image on a full frame of the display image 31 c,and generates a display image where the extracted specific image 33 c isarranged at a specific image detection part of the shrunk whole image.In this case, since a lead line is unnecessary, the display imagebecomes clear.

In addition, as shown in FIG. 17, the display image generation unit 15may generates a display image 31 d, where the shrunk whole image 32 b isdisplayed such that the image 32 b is projected on a screen configuredto face diagonally forward, for example, a lead line 34 d is leaded outtoward a direction perpendicular to the screen from a detection part ofa specific image displayed on the screen, and an extracted specificimage 33 d is displayed at the end of the lead line. In this case, sincethe extracted specific image 33 d and the shrunk whole image 32 d areapparently seen on a different plane in three dimensions, respectively,the extracted specific image 33 d and the shrunk whole image 32 d can beseparated easily.

In all embodiments described above, the extracted specific image 33 dand the shrunk whole image 32 d are displayed on the same display 3.However, the extracted specific image 33 d and the shrunk whole image 32d may be displayed on a different display 3, respectively.

1. An image processing apparatus connected to a monitoring camera and adisplay for processing an image input from the monitoring camera,comprising: an image storage unit for storing an image including aninput image input from the monitoring camera; a characteristic parameterstorage unit for storing a characteristic parameter characterizing aspecific image extracted from the input image; a specific imageextraction unit, wherein the specific image extraction unit cuts outimages having a plurality of predetermined sizes from all parts of theinput image stored in the image storage unit, executes characterevaluation processing for checking whether or not cut out images have anidentical character to the specific image characterized by thecharacteristic parameter for each cut out image, and extracts the cutout images which are determined to have an identical character to thespecific image by the character evaluation processing as an extractionspecific image; and a display image generation unit for generating adisplay image for displaying a whole image of the input image and theextraction specific image on the display, wherein the characteristicparameter storage unit stores a plurality of sets of the characteristicparameter, wherein the specific image extraction unit executes thecharacter evaluation processing for each of the plurality of sets of thecharacteristic parameter stored in the characteristic parameter storageunit, and when the cut out image is determined to have identicalcharacter to the specific image in every executed character evaluationprocessing, extracts the cut out image as the extraction specific image,wherein the image processing apparatus further comprises a characterevaluation processing control unit for setting a priority of theprocessing in regard to the character evaluation processing for theplurality of sets of the characteristic parameter, wherein the specificimage extraction unit executes the character evaluation processing forthe plurality of sets of the characteristic parameter based on thepriority set by the character evaluation processing control unit, aprocessing progress information storage unit for storing processingprogress information of the character evaluation processing, wherein thecharacter evaluation processing control unit stores identificationinformation of the characteristic parameter indicating that thecharacter evaluation processing is not processed in the processingprogress information storage unit when the predetermined limit timeelapses before the specific image extraction unit completes thecharacter evaluation processing for all the plurality of sets of thecharacteristic parameter and the extraction specific image is extracted,reads out the identification information of the characteristic parameterindicating that the character evaluation processing is not processedfrom the processing progress information storage unit when thepredetermined limit time remains after the specific image extractionunit completes the character evaluation processing for all the pluralityof sets of the characteristic parameter, and makes the specific imageextraction unit execute the character evaluation processing for thecharacteristic parameter identified by read out identificationinformation.
 2. The image processing apparatus according to claim 1,wherein when a predetermined limit time, which is set in advance,elapses before completing the character evaluation processing for allthe plurality of sets of the characteristic parameter, the specificimage extraction unit extracts the extraction specific image based on aprocessing result of the character evaluation processing executed beforethe predetermined limit time elapses.
 3. An image processing apparatusconnected to a monitoring camera and a display for processing an imageinput from the monitoring camera, comprising: an image storage unit forstoring an image including an input image input from the monitoringcamera; a characteristic parameter storage unit for storing acharacteristic parameter characterizing a specific image extracted fromthe input image; a specific image extraction unit, wherein the specificimage extraction unit cuts out images having a plurality ofpredetermined sizes from all parts of the input image stored in theimage storage unit, executes character evaluation processing forchecking whether or not cut out images have an identical character tothe specific image characterized by the characteristic parameter foreach cut out image, and extracts the cut out images which are determinedto have an identical character to the specific image by the characterevaluation processing as an extraction specific image; and a displayimage generation unit for generating a display image for displaying awhole image of the input image and the extraction specific image on thedisplay, wherein the display image generation unit generates a shrunkwhole image which is lowered an image Quality as a whole image of theinput image to be generated in the display image, and wherein thedisplay image generation unit generates the shrunk whole image such thatthe shrunk whole image is projected on a screen which is arranged toface diagonally-forward in the display image.
 4. The image processingapparatus according to claim 3, wherein the display image generationunit, further generates a display image where the extraction specificimage is arranged at outside of the shrunk whole image, and a lead lineconnecting a part where the extraction specific image is extracted inthe shrunk whole image and the extraction specific image which isarranged at outside of the shrunk whole image.
 5. The image processingapparatus according to claim 3, wherein the display image generationunit arranges the extraction specific image at a position of a partwhere the extraction specific image is extracted in the shrunk wholeimage.
 6. An image processing method in an image processing apparatusconnected to a monitoring camera and a display for processing an imageinput from the monitoring camera, wherein the image processing apparatuscomprises: an image storage unit for storing an image including an inputimage input from the monitoring camera; and a characteristic parameterstorage unit for storing a characteristic parameter characterizing aspecific image extracted from the input image; wherein the imageprocessing method executes, steps of: a step for extracting a specificimage, wherein the step for extracting the specific image cuts outimages having a plurality of predetermined sizes from all parts of theinput image stored in the image storage unit, executes characterevaluation processing for checking whether or not cut out images have anidentical character to the specific image characterized by thecharacteristic parameter for each cut out image, and extracts the cutout images which are determined to have an identical character to thespecific image by the character evaluation processing as an extractionspecific image; and a step for generating a display image for displayinga whole image of the input image and the extraction specific image onthe display, wherein the image processing apparatus stores a pluralityof sets of the characteristic parameter of the specific image in thecharacteristic parameter storage unit, and in the step for extractingthe specific image, executes the character evaluation processing foreach of the plurality of sets of the characteristic parameter stored inthe characteristic parameter storage unit, and when the cut out image isdetermined to have identical character to the specific image in everyexecuted character evaluation processing, extracts the cut out image asthe extraction specific image, wherein the image processing apparatus,further comprises a processing progress information storage unit forstoring processing progress information of the character evaluationprocessing, and in the step for extracting the specific image, storesidentification information of the characteristic parameter indicatingthat the character evaluation processing is not processed in theprocessing progress information storage unit when the predeterminedlimit time elapses before the specific image extraction unit completesthe character evaluation processing for all the plurality of sets of thecharacteristic parameter and the extraction specific image is extracted,reads out the identification information of the characteristic parameterindicating that the character evaluation processing is not processedfrom the processing progress information storage unit when thepredetermined limit time remains after the specific image extractionunit completes the character evaluation processing for all the pluralityof sets of the characteristic parameter, and executes the characterevaluation processing for the characteristic parameter identified byread out identification information.
 7. The image processing methodaccording to claim 6, wherein in the step for extracting the specificimage, when a predetermined limit time, which is set in advance, elapsesbefore completing the character evaluation processing for all theplurality of sets of the characteristic parameter, the image processingapparatus extracts the extraction specific image based on a processingresult of the character evaluation processing executed before thepredetermined limit time elapses.
 8. The image processing methodaccording to claim 6, wherein the image processing apparatus sets apriority of the processing in regard to the character evaluationprocessing for the plurality of sets of the characteristic parameter,and in the step for extracting the specific image, executes thecharacter evaluation processing for the plurality of sets of thecharacteristic parameter based on the priority set by the imageprocessing apparatus.
 9. An image processing method in an imageprocessing apparatus connected to a monitoring camera and a display forprocessing an image input from the monitoring camera, wherein the imageprocessing apparatus comprises: an image storage unit for storing animage including an input image input from the monitoring camera; and acharacteristic parameter storage unit for storing a characteristicparameter characterizing a specific image extracted from the inputimage; wherein the image processing method executes, steps of a step forextracting a specific image, wherein the step for extracting thespecific image cuts out images having a plurality of predetermined sizesfrom all parts of the input image stored in the image storage unit,executes character evaluation processing for checking whether or not cutout images have an identical character to the specific imagecharacterized by the characteristic parameter for each cut out image,and extracts the cut out images which are determined to have anidentical character to the specific image by the character evaluationprocessing as an extraction specific image; and a step for generating adisplay image for displaying a whole image of the input image and theextraction specific image on the display, wherein in the step forgenerating the display image, the image processing apparatus generates ashrunk whole image which is lowered an image quality as a whole image ofthe input image to be generated in the display image, wherein in thestep for generating the display image, the image processing apparatusgenerates the shrunk whole image such that the shrunk whole image isprojected on a screen which is arranged to face diagonally-forward inthe display image.
 10. The image processing method according to claim 9,wherein in the step for generating the display image, the imageprocessing apparatus, further generates a display image where theextraction specific image is arranged at outside of the shrunk wholeimage, and a lead line connecting a part where the extraction specificimage is extracted in the shrunk whole image and the extraction specificimage which is arranged at outside of the shrunk whole image.
 11. Theimage processing method according to claim 9, wherein in the step forgenerating the display image, the image processing apparatus arrangesthe extraction specific image at a position of a part where theextraction specific image is extracted in the shrunk whole image.